AutoLR: Layer-wise Pruning and Auto-tuning of Learning Rates in Fine-tuning of Deep Networks

Existing fine-tuning methods use a single learning rate over all layers. In this paper, first, we discuss that trends of layer-wise weight variations by using do not match the well-known notion lower-level layers extract general features and higher-level specific features. Based on our discussion, propose an algorithm improves performance reduces network complexity through pruning auto-tuning rates. The proposed has verified effectiveness achieving state-of-the-art image retrieval benchmark datasets (CUB-200, Cars-196, Stanford online product, Inshop). Code is available at https://github.com/youngminPIL/AutoLR.